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DETAILED ACTION 

Election/Restrictions 

1. Applicant's traversal of the Restriction Requirement dated 25 October 2004, is 
persuasive. Therefore, the Restriction Requirement has been withdrawn and a full treatment of 
the claims is found below. 

Claim Rejections - 35 USC § 102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

3. Claims 1-3, 6, 10, 11, 15, and 16 are rejected under 35 U.S.C. 102(b) as being anticipated 
by U.S. Patent 5,557,684 by Wang et al. ("Wang"). 

Regarding claim 1, Wang discloses a method (figure 8) comprising: 

determining at least one cluster from an image comprising at least one segmented area 

(steps 42 and 44: regions of coherent motion are combined into clusters of motion models from 

an image containing distinct objects in motion); 

estimating cluster parameters for the at least one cluster (steps 46-56: affine motion 

information for each cluster is estimated at step 46); and 
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evaluating the at least one cluster, whereby the step of evaluating is performed in order to 
determine whether to modify the at least one cluster (step 50: the clusters are evaluated to 
determine if any have changed; if some have changed, the process returns to step 46 where the 
clusters are modified; if some have not changed, the process continues to step 52). 
Regarding claim 3, Wang discloses the method of claim 1, wherein: 
the step of estimating cluster parameters (steps 46-56) further comprises the steps of: 
assigning pixels from a selected one of the segmented areas to one of the clusters, the 
step of assigning performed until each pixel from a selected one of the segmented areas has been 
assigned to a cluster (step 52: each pixel in the segmented regions is assigned to a clustered 
motion model); 

re-estimating cluster parameters for each of the clusters (step 46: affine motion 
information for each cluster is re-estimated); and 

determining if at least one convergence criterion is met (step 56: it is determined whether 
the pixel assignments have converged into a stable configuration). 

Regarding claim 1 1, Wang discloses segmented areas are determined through 
background- foreground segmentation (e.g. see figure 1, wherein segmented areas are determined 
by segmenting foreground objects from the background using interframe motion). 

Regarding claims 15 and 16, Wang discloses the system and article of manufacture that 
correspond to claim 1 (see e.g. figures 2 and 7). 



Regarding claim 1, Wang discloses a method (figure 8) comprising: 
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determining at least one cluster from an image comprising at least one segmented area 
(steps 42 and 44: regions of coherent motion are combined into clusters of motion models from 
an image containing distinct objects in motion); 

estimating cluster parameters for the at least one cluster (steps 46-50: affine motion 
information for each cluster is estimated at step 46); and 

evaluating the at least one cluster, whereby the step of evaluating is performed in order to 
determine whether to modify the at least one cluster (step 56: the clusters are evaluated to 
determine if any pixel assignments have changed; if some have changed, the process returns to 
step 42 where the clusters are updated). 

Regarding claim 2, Wang discloses: 

the step of estimating cluster parameters further comprises the step of estimating cluster 
parameters for each of the at least one clusters until at least one first convergence criterion is met 
(i.e. estimating updated clusters at step 46 is carried out until the clusters have not changed (i.e. 
convergence is achieved) at step 50); and 

the step of evaluating cluster parameters further comprises the steps of evaluating cluster 
parameters for each of the at least one clusters until at least one second convergence criterion is 
met, and performing the step of estimating if the at least one second convergence criterion is not 
met (step 56: the clusters are evaluated until the pixel assignments to the various clusters have 
not changed (second convergence criterion); if convergence is not met, then the method proceeds 
to perform the estimating step 46). 



Regarding claim 1, Wang discloses a method (figure 8) comprising: 
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determining at least one cluster from an image comprising at least one segmented area 
(steps 40 and 42: regions, or clusters, associated with coherent motion are identified); 

estimating cluster parameters for the at least one cluster (step 42: affine motion 
information for each cluster is estimated); and 

evaluating the at least one cluster, whereby the step of evaluating is performed in order to 
determine whether to modify the at least one cluster (step 44: the clusters are evaluated to 
determine if they should be split or merged). 

Regarding claim 6, Wang discloses the step of evaluating cluster parameters further 
comprises the steps of: 

determining whether a selected cluster should be split; 

splitting the selected cluster into at least two clusters when it is determined that the 
selected cluster should be split (i.e. step 44 determines whether a cluster should be split and if so, 
the cluster is split into two different clusters). 

Regarding claim 10, Wang discloses the step of evaluating the at least one cluster 
comprises splitting a current cluster (see step 44). 

4. Claims 1, 4, and 5 are rejected under 35 U.S.C. 102(e) as being anticipated by U.S. Patent 

6,272,250 by Sun et al. ("Sun"). 

Regarding claim 1, Sun discloses a method (figure 3) comprising: 

determining at least one cluster from an image comprising at least one segmented area 

(step 40: for an initial image frame containing segmented areas (i.e. objects), a first cluster is 
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determined when a condition is met (step 38), and an input vector, which correspond to a pixel, 
is assigned to the cluster); 

estimating cluster parameters for the at least one cluster (step 44: the prototype vector of 
each existing cluster is estimated); and 

evaluating the at least one cluster, whereby the step of evaluating is performed in order to 
determine whether to modify the at least one cluster (step 46: the clusters are analyzed to 
determine if any are outliers and should be deleted). 

Regarding claim 4, Sun discloses determining whether a selected cluster should be 
deleted; and deleting the selected cluster when it is determined that the selected cluster should be 
deleted (step 46, figure 3). 

Regarding claim 5, Sun discloses the step of determining whether a selected cluster 
should be deleted comprises the steps of: determining if the selected cluster encompasses a 
predetermined number of pixels from a segmented area; and determining that the selected cluster 
should be deleted when the selected cluster does not encompasses the predetermined number of 
pixels from a segmented area (see column 5, lines 47-67). 

Claim Rejections - 35 USC § 103 
5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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6. This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

7. Claims 4 and 5 are rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. 
Patent 5,557,684 by Wang et al. ("Wang") in view of U.S. Patent 6,272,250 by Sun et al. 
("Sun"). 

Regarding claim 4, Wang does not disclose determining whether a selected cluster should 
be deleted; and deleting the selected cluster when it is determined that the selected cluster should 
be deleted. 

Sun discloses a system similar to that of Wang wherein objects are tracked over a 
sequence of images. In particular, Wang discloses that detected clusters of pixels are subject to 
an outlier test 46, figure 3. The outlier test determines whether a cluster should be deleted, and if 
so, deletes the cluster. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Wang by Sun to delete clusters, as claimed, since Sun teaches that certain clusters may 
be indicative of noise or outliers and should be deleted (see column 5, lines 47-67). 

Regarding claim 5, Sun discloses the step of determining whether a selected cluster 
should be deleted comprises the steps of: determining if the selected cluster encompasses a 
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predetermined number of pixels from a segmented area; and determining that the selected cluster 
should be deleted when the selected cluster does not encompasses the predetermined number of 
pixels from a segmented area (see column 5, lines 47-67). 

8. Claim 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. Patent 
5,557,684 by Wang et al. ("Wang") in view of U.S. Patent 5,519,789 by Etoh. 
Regarding claim 14, Wang discloses: 

each pixel belonging to a segmented area is a foreground pixel (see figure 1 : each pixel 
belonging to a segmented object is in the foreground); and 

the step of estimating cluster parameters comprises the steps of: 

assigning each foreground pixel to each of the clusters so that a probability that a pixel 
belongs to a selected cluster is maximized (column 9, lines 18-49: each pixel is assigned to the 
motion model (cluster) that exhibits a minimal error (i.e. a maximal probability)); and 

estimating the parameters of each cluster to fit the pixels assigned to a selected cluster 
within a predetermined error (column 9, lines 18-49: the affine motion model of each cluster is 
estimated in order to fit the pixels assigned to the cluster within a minimum error). 

Wang does not disclose that each cluster is an ellipse, and that processing is carried out 
on the basis of ellipses rather than irregular clusters. 

Etoh discloses an image clustering apparatus that detects salient objects, such as people. 
In particular, Etoh discloses that objects formed of a plurality of ellipses and are tracked on the 
basis of the ellipses (see figures 2, 3, 7, and 8). It would have been obvious to one of ordinary 
skill in the art at the time of the invention to modify Wang by Etoh to utilize ellipses to track 
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objects in an image since Etoh teaches that utilizing ellipses is substantially equivalent to using 
irregularly clusters for the purposes of tracking objects since both methods are adapted to 
distinguish and track salient objects. See column 12, lines 1-26. 

9. Claims 6 and 7 are rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. 
Patent 6,272,250 by Sun et al. ("Sun") in view of U.S. Patent 5,585,944 by Rodriguez. 

Regarding claim 6, Sun is silent to determining whether to split a cluster and if so, 
splitting the cluster, as claimed. 

Rodriguez, like Sun, discloses an image processing system that involves forming regions 
of homogeneous color. In particular, Rodriguez discloses determining whether a region is 
homogeneous with respect to color and then splitting the region if it is not (see figures 12(a) and 
12(b)). It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Sun by Rodriguez to split clusters, as claimed, since Rodriguez teaches that regions 
not exhibiting homogeneity of color should be split in order to maintain regions containing all 
pixels that are sufficiently close in color, in accordance with Sun's stated purpose of tracking 
regions by color. 

Regarding claim 7, Rodriguez discloses the step of determining whether a selected cluster 
should be split comprises the steps of: 

determining how many first pixels from a segmented area are within a first region of the 

cluster; 

determining how many second pixels from a segmented area are within a second region 
of the cluster; and 



Application/Control Number: 09/988,946 Page 10 

Art Unit: 2623 

determining that the selected cluster should be split when a ratio of the second pixels and 
the first pixels meets a predetermined number (see column 9, lines 10-22 and 50-62). 

10. Claims 1 1 and 12 are rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. 
Patent 6,272,250 by Sun et al. ("Sun") in view of U.S. Patent 6,771,818 by Krumm et al. 
("Krumm"). 

Regarding claim 11, Sun is silent to segmented areas being determined through 
background-foreground segmentation. 

Krumm discloses a system for tracking objects over a sequence of images, similar to the 
system of Sun. In particular, Krumm discloses initially identifying objects by performing 
background- foreground segmentation (315, figure 3). It would have been obvious to one of 
ordinary skill in the art at the time of the invention to modify Sun by Krumm to segment the 
foreground from the background since Krumm teaches that segmenting foreground objects from 
the background facilitates the process of tracking salient objects (see column 8, lines 2-39). 

Regarding claim 12, Krumm discloses the background-foreground segmentation 
comprises background subtraction (step 315, figure 3). 

1 1 . Claims 1 3 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over U.S. Patent 
6,272,250 by Sun et al. ("Sun") in view of U.S. Patent 6,771,818 by Krumm et al. ("Krumm"), as 
applied to claim 1 1 above, and further in view of Applicant's Admitted Prior Art ("Admission"). 
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Regarding claim 13, neither Sun nor Krumm expressly discloses marking the segmented 
areas, wherein the marking is performed through binary marking, whereby background pixels are 
marked one color and wherein foreground pixels are marked a different color. 

However, by Applicant's Admission, identifying the foreground from the background in 
a segmented image is conventionally accomplished via a binary marking technique, as claimed 
(see "BACKGROUND OF THE INVENTION" section of the present application). It would 
have been obvious to mark the segmented foreground and background as claimed, since 
Admission teaches that such a technique is conventional and facilitates distinguishing the 
foreground and background regions. 

Allowable Subject Matter 
12. Claims 8 and 9 are objected to as being dependent upon a rejected base claim, but would 
be allowable if rewritten in independent form including all of the limitations of the base claim 
and any intervening claims. 

Regarding claim 8, neither Wang nor Sun disclose determining if a new cluster should be 
added by determining how many pixels in the image are not assigned to a cluster coupled with 
adding the unassigned pixels to a new cluster when the number of pixels that are not assigned to 
a cluster meets a predetermined value. 

Regarding claim 9, neither Wang nor Sun disclsoe determining if a new cluster should be 
added by determining how many pixels in the image are not assigned to a cluster coupled with 
performing a connected component algorithm on the unassigned pixels in order to add at least 
one new cluster. 
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Conclusion 

13. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

U.S. Patent 6,782,126 by Rao et al. 

U.S. Patent 6,704,433 by Matsuo et al. 

U.S. Patent 4,945,478 by Merickel et al. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Colin M. LaRose whose telephone number is (703) 306-3489. If 
attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Amelia 
Au, can be reached on (703) 308-6604. The fax phone number for the organization where this 
application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the TC 2600 Customer Service Office whose telephone number is (703) 
306-0377. 

CML 

Group Art Unit 2623 
7 February 2005 




